CLAIMS 
1. An estimation system for estimating an 

2 object state^ characterized by comprising: 

3 image input means for inputting an input image 

4 containing an object whose state is to be estimated, the 

5 state being at least one of a position and posture; 

6 3D shape data storage means for storing 3D 

7 shape data of the object; 

8 comparison image generation means for 

9 generating, as a comparison image, an image containing 

10 the object in a predetermined state by using the 3D 

11 shape data stored in said 3D shape data storage means; 

12 image positional relationship detection means 

13 for detecting, for each sub-region having a 

14 predetermined size in the image, a positional 

15 relationship between the input image and the comparison 

16 image generated by said comparison image generation 

17 means; 

18 correction amount calculation means for 

19 calculating a correction amount of the object state in 

20 the comparison image by using the positional 

21 relationship detected by said image positional 

22 relationship detection means; and 

23 state correction means for correcting the 

24 object state set in comparison image generation by said 

25 comparison image generation means by using the 

2 6 correction amount obtained by said correction amount 
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27 calculation meanS/ thereby calculating a new object 

28 state. 

2. An estimation system for estimating an 

2 object state according to claim 1, characterized by 

3 further comprising state determination means for 

4 determining on the basis of the correction amount 

5 obtained by said correction amount calculation means 

6 whether the object state set by said comparison image 

7 generation means is appropriate, 

8 wherein when it is determined that the object 

9 state is appropriate, the object state set by said 

10 comparison image generation means is output as an 

11 estimation value, and 

12 when it is determined that the object state is 

13 not appropriate, estimation processing including the 

14 comparison image generation processing by said 

15 comparison image generation means, the positional 

16 relationship detection processing by said image 

17 positional relationship detection means, and the 

18 correction amount calculation processing by said 

19 correction amount calculation means is executed again by 

20 setting the new object state calculated by said state 

21 correction means to the predetermined state. 

3. An estimation system for estimating an 

2 object state according to claim 2, characterized in that 

3 said state determination means determines that the 

4 object state is appropriate when the correction amount 
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5 obtained by said correction amount calculation means is 

6 smaller than a predetermined amount, and determines that 

7 the object state is not appropriate when the correction 

8 amount is not smaller than the predetermined amount. 

4. An estimation system for estimating an 

2 object state according to claim 2, characterized by 

3 further comprising: 

4 first similarity calculation means for 

5 calculating a first similarity between the comparison 

6 image and the input image after the estimation 

7 processing is executed again; and 

8 second similarity calculation means for 

9 calculating a second similarity between the comparison 

10 image and the input image before the estimation 

11 processing is executed again, 

12 wherein said state determination means 

13 compares the first similarity with the second 

14 similarity, determines that the object state is not 

15 appropriate when the first similarity is higher than the 

16 second similarity, and determines that the object state 

17 is appropriate when the first similarity is not higher 

18 than the second similarity. 

5. An estimation system for estimating an 

2 object state according to claim 1, wherein 

3 said image input means comprises means for 

4 inputting a moving image containing an object, and 

5 said image positional relationship detection 
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6 means uses a latest frame image of the moving image as 

7 the input image. 

6. An estimation system for estimating an 

2 object state according to claim 1, characterized in that 

3 said comparison image generation means comprises: 

4 means for reproducing a luminance value of an 

5 object surface, which changes depending on an 

6 illumination condition; and 

7 means for generating the comparison image 

8 under an illumination condition close to that for the 

9 input image by using the reproduced luminance value. 

7. An estimation system for estimating an 

2 object state according to claim 6, characterized by 

3 further comprising illumination base image group storage 

4 means for storing an illumination base image group 

5 representing a variation in luminance of the object 

6 surface depending on the illumination condition, 

7 wherein said comparison image generation means 

8 reproduces the luminance value of the object surface by 

9 using the illumination base image group stored in said 
10 illumination base image group storage means. 

8. An estimation system for estimating an 

2 object state according to claim 7, characterized by 

3 further comprising: 

4 3D shape measuring means for measuring the 3D 

5 shape data of the object and reflectance data of the 

6 object surface; and 
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7 illumination base calculation means for 

8 calculating an illumination base image representing the 

9 variation in luminance of the object surface depending 

10 on the illumination condition by using the 3D shape data 

11 and the reflectance data of the object surface which are 

12 measured by said 3D shape measuring means. 

9. An estimation system for estimating an 

2 object state according to claim 8, characterized in that 

3 said illumination base calculation means 

4 calculates a vector group representing the luminance 

5 value of each point of the 3D shape data under a 

6 plurality of illumination conditions, obtains a base 

7 vector group in descending order of eigenvalues by 

8 principal component analysis of the vector group, and 

9 outputs the base vector group as the illumination base 

10 image group, and 

11 said comparison image generation means 

12 obtains, by using the 3D shape data of the object, a 

13 correspondence between each point of the 3D shape data 

14 of the object and a pixel of the image with the object 

15 being in an estimation value at current time, generates, 

16 by using the correspondence, an image illumination base 

17 group in which the illumination base image group is 

18 projected to the image with the object being in the 

19 estimation value, and generates, as the comparison 

20 image, an image nearest to the input image by linear 

21 connection of the image illumination base group. 
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10. An estimation system for estimating an 

2 object state according to claim 1, characterized in that 

3 said correction amount calculation means calculates, as 

4 the correction amount, a 3D motion of the object which 

5 causes a moving amount of an object part corresponding 

6 to each sub-region in the comparison image to be near to 

7 an image displacement distribution by using the 3D shape 

8 data of the object and the image displacement 

9 distribution representing the positional relationship 

10 between the comparison image and the input image for 

11 each sub-region. 

11. An estimation system for estimating an 

2 object state according to claim 1, characterized by 

3 further comprising feature extraction means for 

4 extracting an image feature amount of each of the input 

5 image and comparison image on the basis of luminance 

6 values of the input image and the comparison image 

7 generated by said comparison image generation means, 

8 wherein said image positional relationship 

9 detection means detects the positional relationship 

10 between the input image and the comparison image for 

11 each sub-region on the basis of the image feature amount 

12 extracted by said feature extraction means. 

12 . An estimation method of estimating an 

2 object state, characterized by comprising the steps of: 

3 inputting an input image containing an object 

4 whose state is to be estimated, the state being at least 
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5 one of a position and posture; 

6 generating, as a comparison image, an image 

7 containing the object in a predetermined state by using 

8 3D shape data of the object; 

9 detecting a positional relationship between 

10 the comparison image and the input image for each 

11 sub-region having a predetermined size in the image; 

12 calculating a correction amount of the object 

13 state in the comparison image by using the detected 

14 positional relationship; and 

15 correcting the object state set in comparison 

16 image generation by using the calculated correction 

17 amount, thereby calculating a new object state . 

13. An estimation method of estimating an 

2 object state according to claim 12, characterized by 

3 further comprising the steps of: 

4 determining on the basis of the calculated 

5 correction amount whether the object state set in 

6 comparison image generation is appropriate; and 

7 outputting the object state set in comparison 

8 image generation as an estimation value when it is 

9 determined that the object state is appropriate, 

10 wherein when it is determined that the object 

11 state is not appropriate, estimation processing 

12 including the step of generating the comparison image, 

13 the step of detecting the positional relationship, and 

14 the step of calculating the correction amount is 
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15 executed again by setting the calculated new object 

16 state to the predetermined state. 

14. An estimation method of estimating an 

2 object state according to claim 13, characterized in 

3 that in the determination step, it is determined that 

4 the object state is appropriate when the correction 

5 amount is smaller than a predetermined amount, and it is 

6 determined that the object state is not appropriate when 

7 the correction amount is not smaller than the 

8 predetermined amount . 

15. An estimation method of estimating an 

2 object state according to claim 13, characterized by 

3 further comprising the steps of: 

4 calculating a first similarity between the 

5 comparison image and the input image after the 

6 estimation processing is executed again; and 

7 calculating a second similarity between the 

8 comparison image and the input image before the 

9 estimation processing is executed again, 

10 wherein in the determination step, the first 

11 similarity is compared with the second similarity, it is 

12 determined that the object state is not appropriate when 

13 the first similarity is higher than the second 

14 similarity, and it is determined that the object state 

15 is appropriate when the first similarity is not higher 

16 than the second similarity. 

16. An estimation method of estimating an 
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2 object state according to claim 12, wherein 

3 in the step of inputting the image, a moving 

4 image containing an object is input, and 

5 in the step of detecting the positional 

6 relationship, a latest frame image of the moving image 

7 is used as the input image. 

17. An estimation method of estimating an 

2 object state according to claim 12, characterized in 

3 that the step of generating the comparison image 

4 comprises the steps of: 

5 reproducing a luminance value of an object 

6 surface, which changes depending on an illumination 

7 condition; and 

8 generating the comparison image under an 

9 illumination condition close to that for the input image 
10 by using the reproduced luminance value. 

18. An estimation method of estimating an 

2 object state according to claim 17, characterized in 

3 that in the step of generating the comparison image, the 

4 luminance value of the object surface is reproduced by 

5 using an illumination base image group representing a 

6 variation in luminance of the object surface depending 

7 on the illumination condition. 

19. An estimation method of estimating an 

2 object state according to claim 18, characterized by 

3 further comprising the steps of: 

4 measuring the 3D shape data of the object and 
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5 reflectance data of the object surface; and 

6 calculating an illumination base image 

7 representing the variation in luminance of the object 

8 surface depending on the illumination condition by using 

9 the 3D shape data and the reflectance data of the object 
10 surface. 

20. An estimation method of estimating an 

2 object state according to claim 19, characterized in 

3 that 

4 in the step of calculating the illumination 

5 base image, a vector group representing the luminance 

6 value of each point of the 3D shape data under a 

7 plurality of illumination conditions is calculated, a 

8 base vector group is obtained in descending order of 

9 eigenvalues by principal component analysis of the 

10 vector group, and the base vector group is output as the 

11 illumination base image group, and 

12 in the step of generating the comparison 

13 image, a correspondence between each point of the 3D 

14 shape data of the object and a pixel of the image with 

15 the object being in an estimation value at current time 

16 is obtained by using the 3D shape data of the object, an 

17 image illumination base group in which the illumination 

18 base image group is projected to the image with the 

19 object being in the estimation value is generated by 

20 using the correspondence, and an image nearest to the 

21 input image is generated as the comparison image by 
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22 linear connection of the image illumination base group. 

21. An estimation method of estimating an 

2 object state according to claim 12, characterized in 

3 that in the step of calculating the correction amount, a 

4 3D motion of the object which causes a moving amount of 

5 an object part corresponding to each sub-region in the 

6 comparison image to be near to an image displacement 

7 distribution is calculated as the correction amount by 

8 using the 3D shape data of the object and the image 

9 displacement distribution representing the positional 

10 relationship between the comparison image and the input 

11 image for each sub-region. 

22. An estimation method of estimating an 

2 object state according to claim 12, characterized by 

3 further comprising the step of extracting an image 

4 feature amount of each of the comparison image and input 

5 image on the basis of luminance values of the comparison 

6 image and input image, 

7 wherein in the step of detecting the 

8 positional relationship, the positional relationship 

9 between the input image and the comparison image for 

10 each sub-region is detected on the basis of the image 

11 feature amount. 

23. An estimation program for estimating an 

2 object state, which causes a computer to execute the 

3 steps of: 

4 inputting an input image containing an object 
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5 whose state is to be estimated, the state being at least 

6 one of a position and posture; 

7 generating, as a comparison image, an image 

8 containing the object in a predetermined state by using 

9 3D shape data of the object; 

10 detecting a positional relationship between 

11 the comparison image and the input image for each 

12 sub-region having a predetermined size in the image; 

13 calculating a correction amount of the object 

14 state in the comparison image by using the detected 

15 positional relationship; and 

16 correcting the object state set in comparison 

17 image generation by using the calculated correction 

18 amount, thereby calculating a new object state. 

24. An estimation program for estimating an 

2 object state according to claim 23, which causes the 

3 computer to further execute the steps of: 

4 determining on the basis of the calculated 

5 correction amount whether the object state set in 

6 comparison image generation is appropriate; 

7 outputting the object state set in comparison 

8 image generation as an estimation value when it is 

9 determined that the object state is appropriate; and 

10 executing again estimation processing 

11 including the step of generating the comparison image, 

12 the step of detecting the positional relationship, and 

13 the step of calculating the correction amount by setting 
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14 the calculated new object state to the predetermined 

15 state when it is determined that the object state is not 

16 appropriate. 

25. An estimation program for estimating an 

2 object state according to claim 24, which causes the 

3 computer to execute, as the determination step, the step 

4 of determining that the object state is appropriate when 

5 the correction amount is smaller than a predetermined 

6 amount, and determining that the object state is not 

7 appropriate when the correction amount is not smaller 

8 than the predetermined amount. 

26. An estimation program for estimating an 

2 object state according to claim 24, which causes the 

3 computer to further execute: 

4 the step of calculating a first similarity 

5 between the comparison image and the input image after 

6 the estimation processing is executed again; 

7 the step of calculating a second similarity 

8 between the comparison image and the input image before 

9 the estimation processing is executed again; and 

10 as the determination step, the step of 

11 comparing the first similarity with the second 

12 similarity, determining that the object state is not 

13 appropriate when the first similarity is higher than the 

14 second similarity, and determining that the object state 

15 is appropriate when the first similarity is not higher 

16 than the second similarity. 
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27. An estimation program for estimating an 

2 object state according to claim 23, which causes the 

3 computer to execute: 

4 as the step of inputting the image, the step 

5 of inputting a moving image containing an object; and 

6 as the step of detecting the positional 

7 relationship, the step of using a latest frame image of 

8 the moving image as the input image. 

28. An estimation program for estimating an 

2 object state according to claim 23, which causes the 

3 computer to execute, in the step of generating the 

4 comparison image, the steps of: 

5 reproducing a luminance value of an object 

6 surface, which changes depending on an illumination 

7 condition; and 

8 generating the comparison image under an 

9 illumination condition close to that for the input image 
10 by using the reproduced luminance value. 

29. An estimation program for estimating an 

2 object state according to claim 28, which causes the 

3 computer to execute, as the step of generating the 

4 comparison image, the step of reproducing the luminance 

5 value of the object surface by using an illumination 

6 base image group representing a variation in luminance 

7 of the object surface depending on the illumination 

8 condition. 

30. An estimation program for estimating an 
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2 object state according to claim 29, which causes the 

3 computer to further execute the steps of: 

4 measuring the 3D shape data of the object and 

5 reflectance data of the object surface; and 

6 calculating an illumination base image 

7 representing the variation in luminance of the object 

8 surface depending on the illumination condition by using 

9 the 3D shape data and the reflectance data of the object 
10 surface. 

31. An estimation program for estimating an 

2 object state according to claim 30, which causes the 

3 computer to execute: 

4 as the step of calculating the illumination 

5 base image, the step of calculating a vector group 

6 representing the luminance value of each point of the 3D 

7 shape data under a plurality of illumination conditions, 

8 obtaining a base vector group in descending order of 

9 eigenvalues by principal component analysis of the 

10 vector group, and outputting the base vector group as 

11 the illumination base image group, and 

12 as the step of generating the comparison 

13 image, the step of obtaining a correspondence between 

14 each point of the 3D shape data of the object and a 

15 pixel of the image with the object being in an 

16 estimation value at current time by using the 3D shape 

17 data of the object, generating an image illumination 

18 base group in which the illumination base image group is 
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19 projected to the image with the object being in the 

20 estimation value by using the correspondence, and 

21 generating, as the comparison image, an image nearest to 

22 the input image by linear connection of the image 

23 illumination base group. 

32. An estimation program for estimating an 

2 object state according to claim 23, which causes the 

3 computer to execute, as the step of calculating the 

4 correction amount, the step of calculating, as the 

5 correction amount, a 3D motion of the object which 

6 causes a moving amount of an object part corresponding 

7 to each sub-region in the comparison image to be near to 

8 an image displacement distribution by using the 3D shape 

9 data of the object and the image displacement 

10 distribution representing the positional relationship 

11 between the comparison image and the input image for 

12 each sub-region. 

33. An estimation program for estimating an 

2 object state according to claim 23, which causes the 

3 computer to further execute: 

4 the step of extracting an image feature amount 

5 of each of the comparison image and input image on the 

6 basis of luminance values of the comparison image and 

7 input image; and 

8 as the step of detecting the positional 

9 relationship, the step of detecting the positional 

10 relationship between the input image and the comparison 
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11 image for each sub-region on the basis of the image 

12 feature amount. 
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